Scalable Bayesian Meta-Learning through Generalized Implicit Gradients

Meta-learning owns unique effectiveness and swiftness in tackling emerging tasks with limited data. Its broad applicability is revealed by viewing it as a bi-level optimization problem. The resultant algorithmic viewpoint however, faces scalability issues when the inner-level relies on gradient-based iterations. Implicit differentiation has been considered to alleviate this challenge, but restricted an isotropic Gaussian prior, only favors deterministic meta-learning approaches. This work markedly mitigates bottleneck cross-fertilizing benefits of implicit probabilistic Bayesian meta-learning. novel (iBaML) method not broadens scope learnable priors, also quantifies associated uncertainty. Furthermore, ultimate complexity well controlled regardless trajectory. Analytical error bounds are established demonstrate precision efficiency generalized gradient over explicit one. Extensive numerical tests carried out empirically validate performance proposed method.